An introduction to computing and interpreting Cronbach Coefficient Alpha in SAS

In spite of the ease of computation of Cronbach Coefficient Alpha, its misconceptions and mis-applications are still widespread, such as the confusion of consistency and dimensionality, as well as the confusion of raw Alpha and standardized Alpha. To clarify these misconceptions, this paper will illustrate the computation of Cronbach Coefficient Alpha in a step-by-step manner, and also explain the meaning of each component of the SAS output. INTRODUCTION Reliability can be expressed in terms of stability, equivalence, and consistency. Consistency check, which is commonly expressed in the form of Cronbach Coefficient Alpha (Cronbach, 1951), is a popular method. Unlike testretest for stability and alternate form for equivalence, only a single test is needed for estimating internal consistency. In spite of its ease of computation, misconceptions and misapplications of Cronbach Coefficient Alpha are widespread. The following problems are frequently observed: 1. Assumptions of Cronbach Alpha are neglected by researchers and as a result over-estimation and underestimation of reliability are not taken into consideration. 2. Some researchers believe that the standardized Alpha is superior to the raw Alpha because they believe standardization can normalize skewed data. This problem also reflects the confusion of covariance matrix with correlation matrix. 3. Additionally, some people throw out difficult or easy items based on the simple statistics of each item without taking the entire test into account. 4. Further, when a survey or test contains different latent dimensions, some researchers compute the overall Alpha only and jump to the wrong conclusion that the entire test or survey is poorly written. 5. On the other hand, when a high overall Alpha is obtained, many researchers assume a single dimension and do not further investigate whether the test carries subscales. 6. Several researchers use a pretest as the baseline or as a covariate. However, a low Alpha in the pretest may result from random guessing when the subjects have not been exposed to the treatment (e.g. training of the test content). Judging the reliability of the instrument based on the pretest scores is premature. 7. Last but not least, quite a few researchers adopt a validated instrument but skip computing Cronbach Coefficient Alpha with their sample. This practice makes subsequent meta-analysis of mean difference and Alpha impossible. To clarify these misconceptions, this paper will illustrate the computation of Cronbach Coefficient Alpha in a step-by-step manner, and also explain the meaning of each component of the SAS output. WHICH RELIABILITY INFORMATION SHOULD I USE? One could compute Cronbach Coefficient Alpha, Kuder Richardson (KR) Formula, or Spilt-half Reliability Coefficient to examine internal consistency within a single test. Cronbach Alpha is recommended over the other two for the following reasons: 1. Cronbach Alpha can be used for both binary-type and large-scale data. On the other hand, KR can be applied to dichotomously scored data only. 2. Spilt-half can be viewed as a one-test equivalent to alternate form and test-retest, which use two tests. In spilthalf, you treat one single test as two tests by dividing the items into two subsets. Reliability is estimated by computing the correlation between the two subsets. The drawback is that the outcome is affected by how you group the items. Therefore, the reliability coefficient may vary from group to group. On the other hand, Cronbach Alpha is the mean of all possible spilt-half coefficients that are computed by the Rulon method (Crocker & Algina, 1986). WHAT IS CRONBACH ALPHA? Cronbach coefficient Alpha is a measure of squared correlation between observed scores and true scores. Put another way, reliability is measured in terms of the ratio of true score variance to observed score variance. The theory behind it is that the observed score is equal to the true score plus the measurement error (Y = T + E). For example, I know 80% of the material but my score is 85% because of guessing. In this case, my observed score is 85 while my true score is 80. The additional five points are due to the measurement error. It is assumed that a reliable test should minimize the measurement error so that the error is not highly correlated with the true score. On the other hand, the relationship between true score and observed score should be strong. In addition, it is assumed that the mean of the measurement error should be zero. In other words, the error scores should be random and uncorrelated with each other. Failure of meeting this assumption may lead to an overestimation of Cronbach Alpha though in practice this assumption cannot be fully met. It is also assumed that items must be essentially tau equivalent, in which the true scores for any two items must be within a constant of each other for an examine. If this assumption for Cronbach Alpha is violated, Alpha may underestimate reliability. For this reason, it is generally agreed that Cronbach Alpha is a lower bound estimate of reliability because perfect essentially tau-equivalence is seldom achieved (Cortina, 1993). Using simulations, Zimmerman, and Zumbo (1993) found that the violations of these assumptions lead to substantive over-estimation and under-estimation of Cronbach Alpha. Researchers should be aware of these potential problems while applying Cronbach Alpha. Statistics, Data Analysis, and Data Mining